Analysing and quantifying visual experience in medical imaging

Healthcare professionals increasingly view medical images and videos in a variety of environments. The perception and interpretation of medical visual information across all specialties, career stages, and practice settings are critical to patient care and safety. However, medical images and videos are not self-explanatory and thus need to be interpreted by humans, who are prone to errors caused by the inherent limitations of the human visual system. It is essential to understand how medical experts perceive visual content, and use this knowledge to develop new solutions to improve clinical practice. Progress has been made in the literature towards such understanding, however studies remain limited. This thesis investigates two aspects of human visual experience in medical imaging, i.e., visual quality assessment and visual attention. Visual quality assessment is important as diverse visual signal distortion may arise in medical imaging and affect the perceptual quality of visual content, and therefore potentially impact the diagnosis accuracy. We adapted existing qualitative and quantitative methods to evaluate the quality of distorted medical videos. We also analysed the impact of medical specialty on visual perception and found significant differences between specialty groups, e.g., sonographers were in general more bothered by visual distortions than radiologists. Visual attention has been studied in medical imaging using eye-tracking technology. In this thesis, we firstly investigated gaze allocation with radiologists analysing two-view mammograms and secondly assessed the impact of expertise and experience on gaze behaviour. We also evaluated to what extent state-of-the-art visual attention models can predict radiologists’ gaze behaviour and showed the limitations of existing models. This thesis provides new experimental designs and statistical processes to evaluate the perception of medical images and videos, which can be used to optimise the visual experience of image readers in clinical practice

State of the art: Eye-tracking studies in medical imaging

Eye-tracking – the process of measuring where people look in a visual field – has been widely used to study how humans process visual information. In medical imaging, eye-tracking has become a popular technique in many applications to reveal how visual search and recognition tasks are performed, providing information that can improve human performance. In this paper, we present a comprehensive review of eye-tracking studies conducted with medical images and videos for diverse research purposes, including identification of degree of expertise, development of training, and understanding and modelling of visual search patterns. In addition, we present our recent eye-tracking study that involves a large number of screening mammograms viewed by experienced breast radiologists. Based on the eye-tracking data, we evaluate the plausibility of predicting visual attention by computational models

The impact of specialty settings on the perceived quality of medical ultrasound video

Health care professionals are increasingly viewing medical images and videos in a variety of environments. The perception of medical visual information across all specialties, career stages, and practice settings are critical to patient care and patient safety. Visual signal distortions, such as various types of noise and artifacts arising in medical imaging, affect the perceptual quality of visual content and potentially impact diagnoses. To optimize clinical practice, it is of fundamental importance to understand the way medical experts perceive visual quality. Psychophysical studies have been undertaken to evaluate the impact of visual distortions on the perceived quality of medical images and videos. However, very little research has been conducted on how speciality settings affect the perception of visual quality. In this paper, we investigate whether and how radiologists and sonographers differently perceive the quality of compressed ultrasound videos, via a dedicated subjective experiment. The findings can be used to develop useful solutions for improved visual experience and better image-based diagnoses

Study of video quality assessment for telesurgery

elemedicine provides a transformative practice for access to and delivery of timely and high quality healthcare in resource-poor settings. In a typical scenario of telesurgery, surgical tasks are performed with one surgeon situated at the patient’s side and one expert surgeon from a remote site. In order to make telesurgery practice realistic and secure, reliable transmission of medical videos over large distances is essential. However, telesurgery videos that are communicated remotely in real time are vulnerable to distortions in signals due to data compression and transmission. Depending on the system and its applications, visual content received by the surgeons differs in perceived quality, which may incur implications for the performance of telesurgery tasks. To rigorously study the assessment of the quality of telesurgery videos, we performed both qualitative and quantitative research, consisting of semi-structured interviews and video quality scoring with human subjects. Statistical analyses are conducted and results show that compression artifacts and transmission errors significantly affect the perceived quality; and the effects tend to depend on the specific surgical procedure, visual content, frame rate, and the degree of distortion. The findings of the study are readily applicable to improving telesurgery systems

CUDAS: Distortion-aware saliency benchmark

Visual saliency prediction remains an academic challenge due to the diversity and complexity of natural scenes as well as the scarcity of eye movement data on where people look in images. In many practical applications, digital images are inevitably subject to distortions, such as those caused by acquisition, editing, compression or transmission. A great deal of attention has been paid to predicting the saliency of distortion-free pristine images, but little attention has been given to understanding the impact of visual distortions on saliency prediction. In this paper, we first present the CUDAS database - a new distortion-aware saliency benchmark, where eye-tracking data was collected for 60 pristine images and their corresponding 540 distorted formats. We then conduct a statistical evaluation to reveal the behaviour of state-of-the-art saliency prediction models on distorted images and provide insights on building an effective model for distortion-aware saliency prediction. The new database is made publicly available to the research community